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TECHNIQUE FOR DEFINING CONCEALMENT ORDER TO MINIMIZE ERROR 
\ — . PROPAGATION . i 

TECHNICAL FIELD 

This invention relates to a technique for establishing the order of concealment of 
macroblocks having lost and/or corrupted data. 



BACKGROUND ART 

10 

In many instances, video streams undergo compression (coding) to facilitate storage and 
transmission. Not infrequently, such coded video streams incur data losses or become corrupted 
during transmission because of channel errors and/or network congestion. Upon decoding, the 
loss/corruption of data manifests itself as missing pixel values within one or more macroblocks 
1 5 within the decoded image. To reduce artifacts attributable to such missing/corrupted pixel 

values, a decoder will "conceal" such missing/corrupted pixel values by estimating the values 
from other macroblocks in the same image or from another image. The term conceal is a 
somewhat of a misnomer because the decoder does not actually hide missing or corrupted pixel 
values. 

20 Spatial concealment seeks to derive the missing/corrupted pixel values by using pixel 

values from other areas in the same image in reliance on the similarity between neighboring 
regions in the spatial domain. In contrast to spatial error concealment, temporal concealment 
attempts the recovery of the coded motion information, namely the reference picture indices and 
the motion vectors, to estimate the missing pixel values from at least one previously transmitted 

25 macroblock. When the errors do not affect isolated macroblocks but groups of contiguous 

macroblocks, concealing errors can require using information from already concealed blocks. 
The concealment of lost slices, as defined in the Main Profile of the ISO/ITU H.264 standard for 
video compression, exemplifies a typical situation requiring the use of information from 
previously concealed macroblocks. However, such a strategy tends to propagate errors and 

30. compromises the quality of the restored image. 

One approach to overcoming this difficulty proposes to conceal entire columns of 
macroblocks, progressing inwards from the boundaries of the image towards the center. The 
macroblocks on the left and the macroblocks on the right undergo concealment independently of 
each other. Similarly,, concealment within each column progresses independently up and down 
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until the concealed macroblocks meet in the middle. This approach reduces enror propagation by 
concealing the center of the image, which is typically difficult to predict but is visually 
important, at the end of the process. However, accomplishing concealment in this manner 
reduces the ability to propagate meaningful information from left to right and vice versa. 
Macroblocks on the left side of the image are predicted with no information coming from the 
right and vice versa. Such behavior makes the propagation of diagonal contours across the 
missing region difficult. 

Thus, there is a need for a technique that accomplishes error concealment while 
overcoming the aforementioned drawback. 

BRIEF SUMMARY OF THE INVENTION 

Briefly, in accordance with the present principles, concealment of macroblocks in a 
rectangular array occurs by concealing the macroblocks having missing/corrupted values by 
progressing in the direction of one of the two larger sides of the array. Thus, for example, for a 
rectangular array having a greater width than height, concealment progresses by tows. 
Conversely, for a rectangular array that has a greater height than width, concealment progresses 
by columns. (For a square array, concealment can progress arbitrarily by rows or by columns.) 
The concealment of macroblocks by progressing in the direction of one of the two larger sides of 
the array facilitates the prediction of contours either in the vertical, horizontal or diagonal 
direction, thereby improving the quality of concealment in contrast to prior art concealment 
techniques that progressively conceal in a given direction irrespective of the dimensions of the 
array. 

BRIEF DESCRIPTION OF THE DRAWINGS 

FIGURE 1 illustrates an image partitioned into macroblocks; 

FIGURE 2 illustrates the image of FIG. 1 after undergoing decoding when the image was 
previously coded and transmitted over an error-prone channel, with errors in the channel causing 
the loss of a rectangular array of macroblocks; 

FIGURE 3 illustrates the image of FIG. 2 during error concealment that progresses 
vertically by columns from the outer edges to the center of the image in accordance with the 
teachings of the prior art; 
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FIGURE 4 illustrates the image of FIG. 2 during error concealment that progresses 
horizontally by rows in accordance with the present principles; and 

FIGURE 5 illustrates the image of FIG- 1 after undergoing decoding when the image was 
previously coded and transmitted over an error-prone channel, with errors in the channel causing 
5 the loss of a non-rectangular array of macroblocks; and 

FIGURE 6 illustrates a partition of the non-rectangular array of lost macroblocks of FIG. 
5 into two rectangular sub-regions to enable error concealment in accordance with the present 
principles. 



DETAILED DESCRIPTION 



BJock-based video compression techniques, such as embodied in the proposed ISO/ITU 
H.264 video compression standard, operate by dividing a picture into slices, each slice 
comprising a set of macroblocks or macroblock pairs, with each macroblock coded in accordance 

1 5 with the standard. A macroblock typically comprises a squared region of 1 6x16 pixels. For 
coding purposes, macroblocks can be further partitioned into sub-macroblocks not necessarily 
squared. Each one of the sub-macroblocks can have different coding modes when the 
macroblock is encoded. FIGURE 1 depicts the partitioning of an original picture 100 into 
macroblocks 1 10. Note that the number of macroblocks within the picture 100 can vary 

20 depending on its size. 

In the illustrated embodiment, the picture 100 has a black stripe 120 that extends 
diagonally from the upper left-hand comer to the lower right-hand comer of the picture. The 
remaining portion of the picture 1 00 has a gray level for purposes of illustration. To facilitate 
transmission, the picture 100 will typically undergo coding, typically by any of several well- 

25 known block-based coding techniques, such as the technique described in the proposed ISO/TTU 
H.264 standard. During the transmission of the coded picture 100, one or more macroblocks 1 10 
can become corrupted. In other words, the pixel values carried by such a macroblock 110 will 
become lost or tainted. An example of such a situation appears in FIG. 2 which illustrates the 
picture 100 following decoding. The decoded picture 100 in FIG. 2 has a number of rows of 

30 macroblocks with missing pixel values, all of which lie in a rectangular error region 130. For 
ease of illustration, the macroblocks 110 with missing pixel values appear in white in FIG. 2, as 
contrasted by those macroblocks with intact pixel values which appear either dark or gray, 
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To assure optimal performance, a non-rectangular error region should be divided into 
rectangular sub-regions prior to undergoing concealment. Undertaking error concealment in 
accordance with the present principles on rectangular error regions, such as the error region 130 
of FIG- 4 assures a symmetrical progression from the top-most and the bottom-most row 
5 • (equivalently, from the left-most and the right-most columns)- If the error region has larger 

width than height, division into rectangular sub-regions is done in order that each sub-region has 
the upper-most and the lower-most rows neighboring correctly received rows. Should the error 
region have a larger height than width, division into rectangular sub-regions is ddne in order that 
each sub-region has the left-most and the right-most rows neighboring correctly received 
. columns. 

To better appreciate the process of partitioning a non-rectangular error region, refer to 
FIG. 5, which depicts the picture 100 of FIG. 1 after decoding when the image prior to decoding 
has undergone transmission through an error-prone channel causing resultant errors. Such errors 
manifest themselves in the picture 1 00 of FIG. 5 in the form of a non-rectangular error region 
15 130'. In accordance with the present principles, the non-rectangular error region 130' undergoes 
partitioning into two or more rectangular sub-regions, depicted as sub-regions 135i and 135 2 in 
FIG. 5, with each sub-region having its upper and lower most rows bordering a correctly received 
row of rnacroblocks. Thereafter, the macroblocks in each of the sub-regions 135i and 135 2 
undergo concealment, by alternately scanning the rows of macroblocks in each of the sub-regions 
in opposite directions as depicted by the serpentine arrows within each sub-region. 

The foregoing describes a technique that establishes an order of concealment of 
macroblocks in a rectangular array that facilitates the prediction of contours either in the vertical, 
horizontal or diagonal direction, thereby improving the quality of concealment. 
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